Methods and systems for adjusting a scoring measure based on query breadth

ABSTRACT

Methods and systems for adjusting a scoring measure of a search result based at least in part on the breadth of a previously-executed search query associated with the search result are described. In one described system, a search engine determines a popularity measure for a search result, and then adjusts the popularity measure based at least in part on a query breadth measure of a previously-executed search query associated with the search result. The search engine may use a variety of query breadth measures. For example, the search engine may use the quantity of results returned by the search query, the length of the query, the IR score drop-off, or some other measure of breadth.

FIELD OF THE INVENTION

The present invention relates generally to methods and systems forinformation retrieval. The present invention relates particularly tomethods and systems for adjusting a scoring measure associated with asearch result based on the breadth of a previously-executed search queryassociated with the search result.

BACKGROUND

A conventional network search engine, such as the Google™ search engine,returns a result set in response to a search query submitted by a user.The search engine performs the search based on a conventional searchmethod. For example, one known method, described in an article entitled“The Anatomy of a Large-Scale Hypertextual Search Engine,” by SergeyBrin and Lawrence Page, assigns a degree of importance to a document,such as a web page, based on the link structure of the web page. Thesearch engine ranks or sorts the individual articles or documents in theresult set based on a variety of measures. For example, the searchengine often ranks the results based on a popularity measure. The searchengine generally places the most popular results at the beginning of theresult set.

The popularity measure may comprise one or more individual popularitymeasures. For example, a search engine may utilize the number of times aparticular document has been shown to users, i.e., impression count, asa measure of popularity. A conventional search engine may also use aclick count or click-through ratio as a measure of popularity. Whilethese measures provide valuable information about each result, themeasures can be insufficient, depending on a variety of factors.

A search engine often retrieves a large number of documents for a broadquery. For example, if a user enters a one or two-term query, such as“digital camera,” the search engine is likely to return millions ofresults. Also, many different users may submit this broad queryinitially when searching about material related to digital cameras.Accordingly, the documents returned by these broad queries are oftenover-represented in the popularity counts, and the popularity count foreach one of these results is artificially high because of the number ofbroad queries submitted. Also, documents returned in response to broadqueries are often more abstract than results returned for more specificqueries. The more abstract documents are then over-represented in thepopularity counts, whether based on clicks or based on impressions.

The resulting over-representation of documents due to broad queriestends to skew data collected about the users' behavior. When a userviews a result set from a very broad query, the user will likely seeonly a small fraction of the entire result set. Therefore, it isdifficult to, for example, determine the popularity of documents in theresult set based on the users' response to documents resulting from abroad query.

SUMMARY

Embodiments of the present invention provide methods and systems foradjusting a scoring measure for a search result based at least in parton the breadth of a previously-executed search query associated with theresult. In one embodiment, a search engine determines a popularitymeasure for a search result and adjusts the popularity measure based atleast in part on a query breadth measure of a previously-executed searchquery associated with the search result. The search engine may use avariety of query breadth measures. For example, in one embodiment, thequery breadth measure comprises a quantity of results returned by thesearch query.

These exemplary embodiments are mentioned not to limit or define theinvention, but to provide examples of embodiments of the invention toaid understanding thereof. Exemplary embodiments are discussed in theDetailed Description, and further description of the invention isprovided there. Advantages offered by the various embodiments of thepresent invention may be further understood by examining thisspecification.

BRIEF DESCRIPTION OF THE FIGURES

These and other features, aspects, and advantages of the presentinvention are better understood when the following Detailed Descriptionis read with reference to the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrating an exemplary environment in whichone embodiment of the present invention may operate;

FIG. 2 is a flowchart illustrating a process for associating apopularity measure with a search result in one embodiment of the presentinvention;

FIG. 3 is a flowchart illustrating a process for associating a querybreadth measure with a search query in one embodiment of the presentinvention; and

FIG. 4 is a flowchart illustrating a method for adjusting the popularitymeasure of a result in one embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention comprise methods and systems foradjusting a scoring measure for a search result based at least in parton the breadth of a previously-executed search query associated with thesearch result. In one embodiment, a search engine deweights a popularitymeasure for a result when the value of the popularity measure has beenincreased as a result of the submission of one or more broad queries.The breadth of the query may be calculated in various ways.

Referring now to the drawings in which like numerals indicate likeelements throughout the several figures, FIG. 1 is a block diagramillustrating an exemplary environment for implementation of oneembodiment of the present invention. The system 100 shown in FIG. 1includes multiple client devices 102 a-n in communication with a serverdevice 104 over a network 106. The network 106 shown includes theInternet. In other embodiments, other wired and wireless networks, suchas an intranet may be used. Moreover, methods according to the presentinvention may operate within a single computer.

The client devices 102 a-n shown each includes a computer-readablemedium, such as a random access memory (RAM) 108 coupled to a processor110. The processor 110 executes computer-executable program instructionsstored in memory 108. Such processors may include a microprocessor, anASIC, and state machines. Such processors include, or may be incommunication with, media, for example computer-readable media, whichstores instructions that, when executed by the processor, cause theprocessor to perform the steps described herein. Embodiments ofcomputer-readable media include, but are not limited to, an electronic,optical, magnetic, or other storage or transmission device capable ofproviding a processor, such as the processor 110 of client 102 a, withcomputer-readable instructions. Other examples of suitable mediainclude, but are not limited to, a floppy disk, CD-ROM, DVD, magneticdisk, memory chip, ROM, RAM, an ASIC, a configured processor, alloptical media, all magnetic tape or other magnetic media, or any othermedium from which a computer processor can read instructions. Also,various other forms of computer-readable media may transmit or carryinstructions to a computer, including a router, private or publicnetwork, or other transmission device or channel, both wired andwireless. The instructions may comprise code from anycomputer-programming language, including, for example, C, C++, C#,Visual Basic, Java, Python, Perl, and JavaScript.

Client devices 102 a-n may also include a number of external or internaldevices such as a mouse, a CD-ROM, DVD, a keyboard, a display, or otherinput or output devices. Examples of client devices 102 a-n are personalcomputers, digital assistants, personal digital assistants, cellularphones, mobile phones, smart phones, pagers, digital tablets, laptopcomputers, Internet appliances, and other processor-based devices. Ingeneral, a client device 102 a may be any type of processor-basedplatform that is connected to a network 106 and that interacts with oneor more application programs. Client devices 102 a-n may operate on anyoperating system capable of supporting a browser or browser-enabledapplication, such as Microsoft® Windows® or Linux. The client devices102 a-n shown include, for example, personal computers executing abrowser application program such as Internet Explorer™ from MicrosoftCorporation, Netscape Navigator™ from Netscape CommunicationsCorporation, and Safari™ from Apple Computer, Inc.

Through the client devices 102 a-n, users 112 a-n can communicate overthe network 106 with each other and with other systems and devicescoupled to the network 106. As shown in FIG. 1, a server device 104 isalso coupled to the network 106. In the embodiment shown, a user 112 a-ngenerates a search query 114 at a client device 102 a. The client device102 a transmits the query 114 the server device 104 via the network 106.For example, a user 112 a types a textual search query into a queryfield of a web page of a search engine interface displayed on the clientdevice 102 a, which is then transmitted via the network 106 to theserver device 104. In the embodiment shown, a user 112 a inputs a searchquery 114 at a client device 102 a, which transmits a search querysignal 130 associated with the search query 114 to the server device104. The search query 114 may be transmitted directly to the serverdevice 104 as shown. In another embodiment, the query signal 130 isinstead sent to a proxy server (not shown), which then transmits thequery signal 130 to server device 104. Other configurations arepossible.

The server device 104 shown includes a server executing a search engineapplication program, such as the Google™ search engine. Similar to theclient devices 102 a-n, the server device 104 shown includes a processor116 coupled to a computer-readable memory 118. Server device 104,depicted as a single computer system, may be implemented as a network orcluster of computer processors. Examples of a server device 104 areservers, mainframe computers, networked computers, a processor-baseddevice, and similar types of systems and devices. Client processor 110and the server processor 116 can be any of a number of computerprocessors, such as processors from Intel Corporation of Santa Clara,Calif. and Motorola Corporation of Schaumburg, Ill.

Memory 118 contains the search engine application program, also known asa search engine 120. The search engine 120 locates relevant informationin response to a search query 114 from a user 112 a-n. The search engine120 then provides the result set 134 to the client 102 a via the network106.

In the embodiment shown, the server device 104, or related device, haspreviously performed a crawl of the network 106 to locate articles, suchas web pages, stored at other devices or systems connected to thenetwork 106, and indexed the articles in memory 118 or on another datastorage device. Articles include, for example, web pages of variousformats, such as HTML, XML, XHTML, Portable Document Format (PDF) files,and word processor, database, and application program document files,audio, video, or any other documents or information of any typewhatsoever made available on a network (such as the Internet), apersonal computer, or other computing or storage means. The embodimentsdescribed herein are described generally in relation to HTML files ordocuments, but embodiments may operate on any type of article, includingany type of image.

The search engine 120 includes a document locator 122, a rankingprocessor 124, and a query breadth analyzer 126. In the embodimentshown, each comprises computer code residing in the memory 118. Thedocument locator 122 identifies a set of documents that are responsiveto the search query 114 from a user 112 n by, for example, accessing anindex of documents, indexed in accordance with potential search queriesor search terms. The ranking processor 124 ranks or scores the searchresult 134 including the located set of web pages or documents basedupon relevance to the search query 114 or another criteria, such as apopularity measure. The query breadth analyzer 126 determines orotherwise measures the breadth of the query associated with the querysignal 130. Note that other functions and characteristics of thedocument locator 122, ranking processor 124, and query breadth analyzer126 are further described below.

Server device 104 also provides access to other storage elements, suchas a popularity database 128 and a query breadth database 129. Theserver device 104 can access other similar types of data storagedevices. The popularity database 128 stores measures of the popularityof a document. The popularity database 128 may contain either or bothquery-dependent and query-independent popularity measures. For example,the popularity database 128 may store the number of times a particulardocument has been shown to users—a query-independent measure.Alternatively or additionally, the popularity database 128 may store thenumber of times that the article has been shown to a user in response toa particular keyword—a query-dependent measure. The ranking processor124 utilizes this information in performing rankings of pages in theresult set 134. When the search engine 120 generates a result set 134,the search engine 120 causes a record to be added or modified in thepopularity database 128 to indicate that a particular result was shownto a user 112 a in a result set 134.

Other measures of popularity may be stored in the popularity database128, including, for example, click-through data. Click-through data isgenerally an indicator of quality in a search result. Quality signals orclick-through data can include, but is not limited to, whether aparticular URL or document is clicked by a particular user; how often aURL, document, or web page is clicked by one or more users; how often aparticular user clicks on specific documents or web pages; and the ratioof how often a URL, document, or web page is clicked by one or moreusers to the number of times the URL, document or web page is shown toone or more users (known also as the click-through ratio). A popularitydatabase 128 or similar data storage devices can store other types ofquality signals similar to click-through data, such as any quantitativemeasure of user behavior.

Other data related to documents located in a search result 134 that canbe stored in a popularity database 128 or other data storage device caninclude, but is not limited to, how often a particular URL, document, orweb page is shown in response to a search query 114; how many times aparticular search query 114 is asked by users 112; the age or time aparticular document has been posted on a network 106, and the identityof a source of a particular document on a network 106.

The query breadth database 129 stores measures of query breadth. Variousmeasures of query breadth may be stored. For example, one measure ofquery breadth is the number of results matching a query. The higher thenumber of results, the more likely the query is a broad one.

Another measure of query breadth is the rate at which the informationretrieval (IR) score drops off from a first result until a secondresult, e.g., the first result until the nth result. The terms first andsecond are used here merely to differentiate one item from another item.The terms first and second are not used to indicate first or second intime, or first or second in a list, or other order, unless explicitlynoted. The IR score provides a measure of the relevance of a documentfor a query. If many documents are returned in response to a query, manyof them are likely relevant to the query and will have a correspondinglyhigh IR score. Accordingly, the IR score will drop slowly from oneresult to the next result in the result set. The slower the rate of dropoff, the more likely the query is a broad query. For example, if a queryreturns one million results, the drop off in IR score from the firstelement (IR₁) to the tenth element (IR₁₀) is likely very small. In otherwords, the tenth document is likely to be nearly or just as relevant tothe query as the first result. In contrast, if a query returns only tenresults, the drop off in IR score from the first element to the tenthelement is likely to be very high.

The drop off in IR score may be stored in the query breadth database 129as a ratio, for example, IR₁₀/IR₁. The breadth of the query is assumedto be highest as the ratio approaches 1, where the relevance of thefirst and tenth queries are most nearly equal. The breadth is assumed tobe lowest as the ratio approaches 0. Another related measure of querybreadth is number of results in a result set that have an IR scoregreater than a percentage of the top IR score, e.g., an IR score greaterthan about ninety percent (90%) of the top IR score.

It should be noted that embodiments of the present invention maycomprise systems having different architecture than that which is shownin FIG. 1. For example, in some systems according to the presentinvention, server device 104 may comprise multiple physical servers. Thesystem 100 shown in FIG. 1 is merely exemplary, and is used to explainthe exemplary methods shown in FIGS. 2 and 3.

Various methods may be implemented in the environment shown in FIG. 1and other environments according to the present invention. For examplein one embodiment, a user 112 a enters a search query 114. In response,to receiving the associated query signal 130, a search engine 120locates documents to return in a result set 134. Before returning theresult set 134, the search engine 120 ranks the results. The searchengine 120 may sort or rank the results using a popularity measure. Inone embodiment, the search engine 120 determines a ranking measure, suchas a popularity measure, for a search result in response to the currentquery. The ranking measure may be or may have been adjusted based atleast in part on a query breadth measure of a previously-executed searchquery associated with the search result. In one embodiment, theadjustment is based on the query breadth measures of a plurality ofpreviously-executed search queries associated with the search result.

The search engine 120 may also use measures associated with the searchquery, rather than the number of results returned, to measure thebreadth of the search query. For example, in one embodiment, the searchengine 120 determines the quantity of search terms in a search query.The fewer the quantity of terms in a search query, the broader the queryis likely to be. In another embodiment, the search engine 120 determineshow frequently a specific search query is used. The higher the frequencyof search query use, the broader the query is likely to be. For example,many users 112 a-n are likely to enter a broad query, such as “digitalcamera.” Accordingly, the frequency of use of the query “digital camera”is high.

The search engine 120 may adjust any popularity measure based on thequery breadth measure. The popularity measure that is adjusted may be aquery dependent or query independent measure. A popularity measure thatis query independent is a measure of absolute popularity. For example,in one embodiment, a click-through ratio is query dependent since ittakes into account the number of times the result is shown to a user 112a as a result of a search query 114. In contrast, the click count isquery independent since it is purely based on the number of times anarticle is clicked.

FIG. 2 is a flowchart illustrating a process for associating apopularity measure with a search result in one embodiment of the presentinvention. In the embodiment shown, the search engine 120 receives asignal comprising a result, such as an article, and a popularity measureassociated with the result at 202. The search engine 120 or othercomponent determines whether or not a popularity measure has beenassociated with the result by, for example, searching the popularitydatabase (128) at 204.

If the popularity measure exists for the result, the search engine 120updates the popularity measure, e.g., the search engine may increment aclick count associated with a search result at 206. If the popularitymeasure does not exist, the search engine 120 creates a popularitymeasure/result association in the popularity database (128) at 208. Theprocess shown in FIG. 2 then ends at 210. The popularity measure can beused in subsequent searches to rank a result returned in response to asearch query. The updates to the popularity database 128 may occuroffline so that a user is not affected adversely by the storing orupdating of the popularity measure.

FIG. 3 is a flowchart illustrating a process for associating a querybreadth measure with a search query in one embodiment of the presentinvention. In the embodiment shown, the query breadth analyzer 126receives a query signal at 302. In response, the query breadth analyzerdetermines the breadth of the query at 304. The query breadth analyzer126 may determine the breadth of the query by various methods. Forexample, in one embodiment, the breadth of the query is determined asthe ratio of the IR score of the tenth result returned by the querydivided by the IR score of the first result returned by the query. Inanother embodiment, the breadth of the query is determined by evaluatingthe number of results that are returned by the query. A higher numberindicates a broader query.

In one embodiment, the query breadth analyzer 126 receives theinformation retrieval (IR) score of the first result (IR₁) of the query.The query breadth analyzer 126 also receives the IR score of the nthresult (IR_(n)) of the query. In the embodiment shown in FIG. 3, thevariable n is set based on previously observed results from queries thatare determined in some way to be broad or narrow. The value of n is setto represent the number of results expected from a relatively narrowquery. A higher value of n will result in a fewer number of queriesbeing classified as broad, i.e., a fewer number of queries having a slowIR drop rate. In the embodiment shown, n equals 10. The query breadthanalyzer 126 next computes the ratio of IR_(n) to IR₁ (IR_(n)/IR₁) toarrive at a query breadth measure. To de-weight a result, the computedratio is subtracted from 1 and multiplied by the popularity measure.

If the IR score for the query drops off slowly, IR_(n), will be nearlyequal to IR₁. Accordingly, the ratio of IR_(n), to IR₁ will approach 1and the result of subtracting the ratio from 1 will approach 0. When thepopularity measure is multiplied by 1 minus the breadth measure as shownin the embodiment illustrated in FIG. 4, the popularity measure will bereduced, i.e., deweighted. Accordingly, results that have unnecessarilyhigh popularity measures as a result of an association with one or morebroad queries will be rated less highly in result sets. The querybreadth analyzer 126 next associates the measure of breadth with thequery, e.g., the query breadth analyzer 126 stores the association inthe query breadth database (129) at 306.

Embodiments of the present invention may utilize query-dependent orquery-independent measures for determining the popularity of a result. Aquery-dependent measure is a measure of popularity that is dependent onthe query that returned a result. A query-independent measure provides ameasure of absolute popularity for a result, regardless of the contextin which the user is presented the result. For example, one embodimentutilizes the click-through ratio for a page to determine the granularityof a page. A very low click-through ratio implies that a result isretrieved in response to many broad queries but is not particularlyrelevant to users' questions. Accordingly, when the ratio of clicks toimpressions is very low, one such embodiment deweights the popularitymeasure based on the impressions. In another embodiment, the click countis utilized as the measure of popularity. The click count simplymeasures the number of times an article is clicked without reference tothe number of times the article is shown to the user 112 a.

An embodiment of the present invention may utilize a variety of measuresof query breadth that are not directly related to the actual results.Such measures imply the breadth of the query. For example, in oneembodiment, a simple measure of breadth, the number of terms used in thesearch query, provides the measure of query breadth. In general, thefewer the number of terms in the query, the broader the query is. In onesuch embodiment, the search engine 120 calculates the ratio of thelength of the query to the average length of search queries. The lowerthe ratio, the broader the query.

In another embodiment of the present invention, the search engine 120determines the number of times that the exact or almost exact searchquery has been issued by users 112 a-n. The higher the frequency, themore likely the query is to be a broad query. The length of the searchquery and the frequency of submission of the search query imply that thequeries are broad. However, a query may be short or frequent and stillrelatively narrow. For example, many users may issue a query for aunique surname that has been present in the media. These queries mayresult in a small number of documents in the result set 134. Therefore,in various embodiments of the present invention, multiple measures ofquery breadth are combined to determine the breadth of the query andadjust the popularity measure for the search query.

FIG. 4 is a flowchart illustrating a method for adjusting the popularitymeasure of a result in one embodiment of the present invention. In theembodiment shown, the query breadth analyzer 126 retrieves a record froma log file or other data store that comprises a search query and anidentifier of a result that was clicked at 402. The query breadthanalyzer 126 then determines a popularity measure for the pair at 404.For example, for a click count, the query breadth analyzer 126determines how many occurrences of the query/clicked result pair are inthe log. The number of occurrences corresponds to the number of timesthe result was clicked after being retrieved in response to the query.The total number of clicks for the result is a popularity measurecorresponding to the search query.

The query breadth analyzer 126 next determines the breadth of the queryat 406. The query breadth analyzer 126 uses the breadth of the query tode-weight the popularity measure corresponding to the search query at408. The query breadth analyzer 126 stores the de-weighted popularitymeasure in a data store at 410. The query breadth analyzer 126 or othercomponent may repeat the exemplary process shown in FIG. 4 numeroustimes. For example, the process shown may be repeated for each pair ofquery/clicked result found in the data store and/or for each popularitymeasure.

For example, in one embodiment, a user 112 a enters a search query 114comprising the terms “digital camera.” The client 102 a converts thetextual search query 114 into a query signal 130 and transmits the querysignal 130 over the network 106 to the server device 104. The serverdevice 104 executes the search engine 120 and passes the query signal130 to the search engine 120. The search engine 120 locates severalmillion documents satisfying the query, “digital camera,” whichindicates that the query is a broad query. The query breadth analyzer126 stores the query, “digital camera,” along with the count of thenumber of results returned in response to the query in the query breadthdatabase 129. Since the query is broad, many users are likely to enterthe same query. The number of times the same query is executed may alsobe stored in the query breadth database 129.

In an attempt to narrow the result set, the user 112 a subsequentlyenters a search query 114 comprising the terms “canon powershot digitalcamera.” The client 102 a converts the textual search query 114 into aquery signal 130 and transmits the query signal 130 over the network 106to the server device 104. The server device 104 executes the searchengine 120 and passes the query signal 130 to the search engine 120. Thesearch engine 120 locates several hundred thousand documents satisfyingthe query. The query breadth analyzer 126 stores the query and the countof the results returned in response to the query in the query breadthdatabase 129. Since the subsequent query returns a smaller number ofresults, the breadth of the query as measured by the number of documentsreturned for the query “canon powershot digital camera” is smaller thanthe breadth of the query “digital camera.”

In, for example, a nightly batch process executed after the two queriesdescribed above have been submitted, the ranking processor 124 adjuststhe popularity or other ranking measure of a result associated with thepreviously-executed search queries based on the number of resultsreturned in response to each of the two queries. Although, in theexample described, the adjustment is based on only two queries, often,the adjustment is based on a large number of queries.

Since many of the documents resulting from the broadest exemplary query,“digital camera,” will have been shown many times to many users, thedocuments are likely to have high popularity measures. However, thepopularity measure is artificially high; the score would not be as highif not for the execution of the broad queries. Accordingly, the querybreadth analyzer 126 deweights the popularity measures for thesedocuments.

The adjustment may be based solely on the query breadth measure or maybe based only in part on the query breadth measure. For example, theranking processor may also utilize the age of the popularity data inmaking the adjustment. Once the popularity measure of the results havebeen adjusted.

Subsequently, a user enters another search query, e.g., “canon powershotdigital camera Atlanta.” The search engine 122 executes the search queryand retrieves search results. The ranking processor 124 uses thedeweighted popularity measures to sort the results, and the searchengine 120 transmits the sorted result set 134 to the client 102 a

The foregoing description of embodiments of the invention has beenpresented only for the purpose of illustration and description and isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed. Numerous modifications and adaptations thereof will beapparent to those skilled in the art without departing from the spiritand scope of the present invention.

1. A method comprising: estimating, by a server device implemented as asingle computer system or as a network or cluster of computerprocessors, a breadth of a search query; identifying user interactionwith a first document in a result set that is responsive to the searchquery; changing, by the server device, a ranking of a popularity of thefirst document based at least in part on the user interaction with thefirst document and the breadth of the search query, wherein an amount ofthe change in the ranking of the popularity decreases with increasedbreadth of the search query; and making, by the server device, theranking of the popularity of the first document available for respondingto a subsequent search query.
 2. The method of claim 1, whereinestimating the breadth of the search query comprises estimating thebreadth based on a total number of documents in a result set that isresponsive to the search query.
 3. The method of claim 1, whereinestimating the breadth of the search query comprises estimating thebreadth of the search query based on differences in relevances ofdocuments in the result set.
 4. The method of claim 1, whereinestimating the breadth of the search query comprises comparing rates atwhich the documents in the result set are retrieved.
 5. The method ofclaim 1, wherein changing the ranking of the popularity of the firstdocument comprises weighting the user interaction with the firstdocument based on the breadth of the search query.
 6. The method ofclaim 5, wherein changing the ranking of the popularity of the firstdocument further comprises adding the weighted user interaction to apopularity database configured to store measures of a popularity ofdocuments.
 7. The method of claim 1, wherein identifying userinteraction with the first document comprises determining a click countfor the first document.
 8. The method of claim 1, wherein identifyinguser interaction with the first document comprises determining aclick-through ratio for the first document.
 9. The method of claim 1,wherein identifying the user interaction with the first documentcomprises identifying the user interaction independent of a searchquery.
 10. The method of claim 1, further comprising responding, by theserver device, to a subsequent search query based at least in part onthe ranking of the popularity of the first document.
 11. The method ofclaim 10, wherein responding to the subsequent search query comprisesadjusting a ranking of documents in the response to the subsequentsearch query based at least in part on the ranking of the popularity ofthe first document.
 12. The method of claim 1, wherein changing theranking of the popularity of the first document comprising increasingthe ranking of the popularity of the first document.
 13. An articlecomprising one or more machine-readable data storage media storinginstructions operable to cause one or more machines to performoperations comprising: estimating a breadth of a search query;identifying user interaction with a first document in a result set thatis responsive to the search query; changing a ranking of a popularity ofthe first document based at least in part on the user interaction withthe first document and the breadth of the search query, wherein anamount of the change in the ranking of the popularity decreases withincreased breadth of the search query; and making the ranking of thepopularity of the first document available for responding to asubsequent search query.
 14. The article of claim 13, wherein estimatingthe breadth of the search query comprises estimating the breadth basedon a total number of documents in a result set that is responsive to thesearch query.
 15. The article of claim 13, wherein estimating thebreadth of the search query comprises estimating the breadth of thesearch query based on differences in relevances of documents in theresult set.
 16. The article of claim 13, wherein estimating the breadthof the search query comprises comparing rates at which the documents inthe result set are retrieved.
 17. The article of claim 13, whereinchanging the ranking of the popularity of the first document comprisesweighting the user interaction with the first document based on thebreadth of the search query.
 18. The article of claim 17, whereinchanging the ranking of the popularity of the first document furthercomprises adding the weighted user interaction to a popularity databaseconfigured to store measures of a popularity of documents.
 19. Thearticle of claim 13, wherein identifying user interaction with the firstdocument comprises determining a click count for the first document. 20.The article of claim 13, wherein identifying user interaction with thefirst document comprises determining a click-through ratio for the firstdocument.
 21. The article of claim 13, wherein identifying the userinteraction with the first document comprises identifying the userinteraction independent of a search query.
 22. The article of claim 13,further comprising responding to a subsequent search query based atleast in part on the ranking of the popularity of the first document.23. The article of claim 22, wherein responding to the subsequent searchquery comprises adjusting a ranking of documents in the response to thesubsequent search query based at least in part on the ranking of thepopularity of the first document.
 24. The article of claim 13, whereinchanging the ranking of the popularity of the first document comprisingincreasing the ranking of the popularity of the first document.
 25. Asystem comprising: a server device implemented as a single computersystem or as a network or cluster of computer processors, the serverdevice programmed to perform operations, the operations comprising:estimating a breadth of a search query; identifying user interactionwith a first document in a result set that is responsive to the searchquery; changing a ranking of a popularity of the first document based atleast in part on the user interaction with the first document and thebreadth of the search query, wherein an amount of the change in theranking of the popularity decreases with increased breadth of the searchquery; and making the ranking of the popularity of the first documentavailable for responding to a subsequent search query.
 26. The system ofclaim 25, wherein the server device executes a search engine applicationprogram.
 27. The system of claim 25, wherein the system furthercomprises: a data communication network coupled to the server device;and a data processing device programmed to transmit the search queryover the data communication network to the server device.
 28. The systemof claim 27, wherein the data processing device comprises a clientdevice with a search engine interface having a query field..
 29. Thesystem of claim 27, wherein the data processing device comprises a proxyserver.
 30. The system of claim 27, wherein estimating the breadth ofthe search query comprises estimating the breadth based on a totalnumber of documents in a result set that is responsive to the searchquery.
 31. The system of claim 27, wherein estimating the breadth of thesearch query comprises estimating the breadth of the search query basedon differences in relevances of documents in the result set.
 32. Thesystem of claim 27, wherein estimating the breadth of the search querycomprises comparing rates at which the documents in the result set areretrieved.
 33. The system of claim 27, wherein changing the ranking ofthe popularity of the first document comprises weighting the userinteraction with the first document based on the breadth of the searchquery.
 34. The system of claim 33, wherein changing the ranking of thepopularity of the first document further comprises adding the weighteduser interaction to a popularity database configured to store measuresof a popularity of documents.
 35. The system of claim 27 whereinidentifying user interaction with the first document comprisesdetermining a click count for the first document.
 36. The system ofclaim 27, wherein identifying user interaction with the first documentcomprises determining a click-through ratio for the first document. 37.The system of claim 27, wherein identifying the user interaction withthe first document comprises identifying the user interactionindependent of a search query.
 38. The system of claim 27, wherein theoperations further comprise responding to a subsequent search querybased at least in part on the ranking of the popularity of the firstdocument.
 39. The system of claim 38, wherein responding to thesubsequent search query comprises adjusting a ranking of documents inthe response to the subsequent search query based at least in part onthe ranking of the popularity of the first document.
 40. The system ofclaim 27, wherein changing the ranking of the popularity of the firstdocument comprising increasing the ranking of the popularity of thefirst document.